Image pickup apparatus with white balance control based on brightness

ABSTRACT

An image pickup apparatus such as a video camera has the capability of automatic white balance control whereby the white balance is quickly adjusted for any object even just after an operator has turned on a power supply. An image pickup element converts an optical image of an object into an electrical image signal. The image signal is applied to a signal processing circuit and separated into a luminance signal and color signals. The separated color signals or the R and B signals are amplified by gain control circuits under the control of the correction signals supplied by a correction signal computing part, and then applied to a color-difference signal forming circuit which in turn produces color-difference signals (R−Y) and (B−Y). An encoder circuit generates a television signal in the standard form from the above color-difference signals and the luminance signal. In this processing, the white balance is adjusted according to a white balance control signal generated on the basis of information about a state of illuminance onto an object picked up last time and stored in a data storing part.

This application is a continuation of application Ser. No. 08/424,811,filed Apr. 19, 1995, now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup apparatus such as avideo camera.

2. Description of the Related Art

In conventional image pickup apparatus such as video cameras, one knowntechnique of automatic white balance adjustment is to perform feedbackcontrol such that, for example, color-difference signals become zero.

Under usual conditions where the image of an object is being picked upcontinuously, a state of illumination onto the object correctly reflectsthe image pickup operation. However, in some cases, a state ofillumination onto an object cannot be identified. Such a problem occursfor example in a certain time period just after the power has beenturned on. In this case, the white extraction should be performed over awide range since there are various possibilities regarding the type oflight source illuminating the object. The expansion of the whiteextracting range for a certain time period after an image pickupoperation has been started causes an increase in time required tocorrectly adjust the white balance.

SUMMARY OF THE INVENTION

In view of the above problem, it is an object of the present inventionto provide an image pickup apparatus having the capability of automaticwhite balance adjustment whereby the white balance can be quickly andcorrectly adjusted for an arbitrary object even just after the powersupply has been turned on.

According to an aspect of the invention to achieve the above object,there is provided an image pickup apparatus comprising: image pickupmeans for converting an optical image of an object into an electricalimage signal; a signal processing circuit for recording the image signalfrom the image pickup means; storage means for storing information abouta state of the object obtained in process of recording; and controlmeans for comparing a state of an object obtained immediately afterturning on a power supply with the information stored in the storagemeans and for controlling an image pickup condition according to acomparison result.

According to another aspect of the invention, there is provided an imagepickup apparatus comprising: image pickup means for picking up an imageof an object; display means for displaying an output of the image pickupmeans; detecting means for detecting whether the object is beingobserved via the display means; storage means for storing a state of theobject in response to an output of the detecting means; and controlmeans for comparing a state of an object obtained immediately afterturning on a power supply with the state of the object stored in thestorage means and for controlling an image pickup condition according toa comparison result.

With the above arrangements according to the present invention, it isdetermined whether a new object to be recorded just after turning on thepower supply is identical to the object that was recorded in theprevious operation just before the power supply was turned off, and ifthe new object is almost identical to the previous object, an imagepickup condition can be determined quickly by referring to the previousimage pickup condition obtained just before the power supply was turnedoff.

Other objects and aspects of the invention will become more apparentfrom the following description of embodiments with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment according to the presentinvention;

FIG. 2 is a schematic diagram illustrating a white extracting range foran indoor scene;

FIG. 3 is a schematic diagram illustrating a white extracting range foran outdoor scene;

FIG. 4 is a schematic diagram illustrating a white extracting range foran object which cannot be identified whether the object is situatedoutdoors or indoors;

FIG. 5 is a schematic diagram illustrating the output of an irisposition detector;

FIG. 6 is a flow chart illustrating a process relating to the embodimentof the invention;

FIG. 7 is a flow chart illustrating a process relating to adiscrimination part; and

FIG. 8 is a block diagram illustrating another embodiment according tothe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a block diagram illustrating a video camera provided withautomatic white balance adjustment means, according to the presentinvention.

In FIG. 1, reference numeral 1 denotes a lens for forming an image of anobject, reference numerals 2 denotes an iris, reference numeral 3denotes an image pickup element for photoelectrically converting anoptical image of the object to an electric signal wherein the opticalimage is formed via the lens 1 and the iris 2, reference numeral 4denotes an AGC amplifier (automatic gain control circuit) provided witha feedback loop for amplifying the output signal of the image pickupelement 3 to a proper signal level, reference numeral 5 denotes a signalprocessing circuit for generating a luminance signal and color signalsfrom the image signal which is picked up by the image pickup element 3and amplified by the AGC amplifier 4, reference numerals 6 and 7 denotegain control circuits for controlling the gains associated with red andblue signals R and B which are two of the above color signals, referencenumeral 8 denotes a color-difference signal forming circuit for formingcolor-difference signals (R−Y) and (B−Y) from the output signal of thesignal processing circuit 5, reference numeral 9 denotes an encodercircuit for converting the luminance signal and the color-differencesignals to a television signal in a standard form and then outputtingthe resulting television signal, and reference numeral 10 denotes acorrection signal computing part for computing a correction signal foruse in the white balance control and outputting the resulting correctionsignal, wherein the correction signal computing part 10 includes acomputation part 11 for performing a basic computation associated withthe white balance control and a discrimination part 12 fordiscriminating a state of illumination onto the object.

Furthermore, reference numeral 20 denotes an iris position detector fordetecting the position of the iris 2, reference numeral 21 denotes anAGC gain detector for detecting the gain of the AGC amplifier 4,reference numeral 22 denotes an image pickup element drive part fordriving the image pickup element 3, reference numeral 23 denotes asynchronizing signal generating part for generating a synchronizingsignal to control the timing of the computing operation of thecorrection signal computing part 10, and reference numeral 24 denotes areference value generating part for generating white balance referencevalues Rref and Bref according to which the correction signal iscomputed.

In FIG. 1, there are also shown color-difference signals (B−Y) and (R−Y)denoted by reference numerals 30 and 31 and a luminance signal Y denotedby reference numeral 32, which are applied to the correction signalcomputing part 10 for use in the white balance adjustment, a B gaincorrection signal 33 and an R gain correction signal 34 which aregenerated by the correction signal computing part 10 and applied to thegain control circuits 6 and 7, respectively, an iris position signal 35generated by the iris position detector 20, a driving signal 36 whichcontrols the image pickup element driving part 22, and an AGC gainsignal 37 generated by the AGC gain detector 21.

Furthermore, in FIG. 1, reference numeral 40 denotes a video signalrecording part for recording a video signal generated from the output ofthe signal processing circuit 5 via the encoder circuit 9, and referencenumeral 41 denotes an operation part via which the starting and stoppingor turning-on and turning-off operations of the video signal recordingpart 40 are controlled.

In the video camera configured in the above-described manner, an opticalimage formed on the image pickup element 3 is converted to an electricalsignal. The image signal is then amplified by the AGC amplifier 4 to aproper signal level and applied to the signal processing circuit 5. Thesignal processing circuit 5 generates a high-frequency component YH ofthe luminance signal, a low-frequency component YL of the luminancesignal, a red signal R, and a blue signal B. The signals R and B areapplied to the gain control circuits 6, 7, respectively, and amplifiedunder the control of the correction signals 33 and 34 supplied by thecorrection signal computing part 10, thereby adjusting the whitebalance. The resultant color signals R′ and B′ are applied to thecolor-difference signal forming circuit 8 and the luminance signal YL isalso applied to the color-difference signal forming circuit 8. Thecolor-difference signal forming circuit 8 forms color-difference signals(R−Y) and (B−Y) from these input signals.

The color-difference signals (R−Y) and (B−Y) as well as the luminancesignal YH are applied to the encoder circuit 9 which in turn generates atelevision signal in the standard form. The color-difference signals(R−Y) and (B−Y) are also applied to the correction signal computing part10. The correction signal computing part 10 performs a basic calculationrequired for controlling the white balance in such a manner that theinput color-difference signals (R−Y) 31 and (B−Y) 30 are compared withthe reference values Rref and Bref corresponding to white wherein thereference values are supplied by the reference value generating part 24,thereby obtaining white balance correction data. The correction data orR and B gain correction signals 34 and 33 calculated by the correctionsignal computing part 10 are applied to the gain control circuits 6 and7, respectively, so that the white balance is correctly controlled.

In the computation of the correction signal, the chromatic effects areavoided by extracting a color close to white from the signals R−Y, B−Y,YH supplied by the signal processing circuit 5 and the operation iscontrolled so that the average of these signals becomes zero. To avoidundesirable chromatic effects on the white balance, undesirable colorcomponents are detected and the white balance is corrected using aweighted average value of these color components.

This means that the white balance is adjusted while taking into accountthe color and luminance components of the entire picture. In thistechnique, the discrimination part 12 determines, from a state ofillumination onto an object, whether the object is situated outdoorswhere the color temperature is relatively high or situated indoors wherethe color temperature is low. This leads to an improvement in whiteextraction accuracy. For example, in the case of an outdoor objecthaving a high color temperature, the white color range is located nearblue as shown in FIG. 3. On the other hand, in the case of an indoorobject having a low color temperature, the white color range is locatednear orange as shown in FIG. 2.

Reference numeral 42 denotes a data storing part for holding informationabout a state of illumination onto an object, etc., obtained in processof the image pickup operation, by storing information associated withsignals 30-37 computed in the correction signal computing part 10.

The operation part 41 shown in FIG. 1 includes object detecting meansfor detecting whether an object whose image is being picked up is acorrect one that an operator intends to pick up, and control means forcontrolling the operation of the data storing part 42 at the beginningor the end of the recording operation of the video signal recording part40.

The information about a state of illumination onto an object includes awhite balance control signal generated from the color signals suppliedby the signal processing circuit 5, a luminance (level) signal, andinformation about illuminance of the object such as the iris position,automatic gain control, and electronic shutter conditions.

The operation of the video camera according to the invention will bedescribed below. A picked-up image signal obtained by the photoelectricconversion performed by the image pickup element 3 is applied to thesignal processing circuit 5 and separated into a luminance signal andcolor signals. The white balance is then adjusted, and the encodercircuit 9 finally generates a television signal in the standard form,wherein the television signal is recorded by the video signal recordingpart 40.

In FIG. 1, the iris position detector 20 such as a Hall element producesan output voltage in such a manner that the output voltage becomeshighest when the iris is fully open and lowest when the iris is fullyclosed. The AGC gain detector 21 detects an AGC feedback voltage of theAGC amplifier 4 and generates an output signal representing the gaincontrol state. The image pickup element drive part 22 is controlled bythe correction signal computing part 10 in such a manner that the chargeaccumulation time of the image pickup element 3 is varied so thatexposure is controlled properly by an electronic shutter operation. Thismeans that the correction signal computing part 10 knows the variousamounts of control regarding the iris position, the AGC, and theelectronic shutter, etc., and thus can identify the illuminance of theobject.

The computation part 11 compares the color-difference signals (R−Y) 31and (B−Y) 30 with the reference values Rref and Bref corresponding towhite, and calculates the white balance correction data from thecomparison results. Thus, the computation part 11 outputs the R gaincorrection signal 34 and the B gain correction signal 33 which are usedto control the white balance. When the signals (R−Y) 31 and (B−Y) 30 areequal to the reference values (Rref, Bref) 24, the color temperature ofan object can be determined.

In general, when an object is situated outdoors and illuminated withsunlight, the object has a high illuminance and a high colortemperature. In contrast, when an object is situated indoors andilluminated with light emitted by an artificial light source, the objecthas a rather low illuminance and a low color temperature. Therefore, ifa state of illumination onto the object is known, it is possible toclassify the light sources into a few types including the outdoor andindoor light sources.

Thus, the discrimination part 12 can roughly identify the light sourcejudging from information about the state of illumination onto theobject, thereby facilitating the white balance adjustment and the whiteextracting range computation by the computation part 11.

As in conventional video cameras, an operator can control via theoperation part 41 the operations such as turning on and off of the powersupply and starting and stopping of recording so that the video signalrecording part 40 can record a video signal generated by the encodercircuit 9.

FIG. 6 is a flow chart of the process according to the invention,illustrating the timing of the operation of storing various items ofinformation in the data storing part 42 according to the operation ofthe operation part 41. In step 51, the power supply is turned on, andthen recording is started in step 52. In step 53, the recording isstopped and the related data is held (stored). In a case where recordingis performed a plurality of times, the data is held for each recordingoperation. When, in step 54, the power supply is turned off, the abovedata is stored in step 55.

The above-described data is held even after the main power supply isturned off, so that the data can be used when the power supply is turnedon next time. In the above example, the data is held at the end of arecording operation. Alternatively, the data may also be held at thebeginning of a recording operation or in the middle of the recordingoperation. The data to be held includes such data representing theilluminance and color temperature of an object. More specifically, thedata includes the information on the iris position, the AGC conditions,the electronic shutter conditions, the R and B gain correction signals,the color-difference signals, and the luminance signal. Information onthe illumination modified or derived from the above information may alsobe employed.

The information to be held should correspond exactly to the object anoperator intends to pick up. When an outdoor scene is picked up, theground, for example, may be picked up unintensionally. If the datarelating to the scene of the ground is stored, then the data willrepresent a dark illuminance and a color which will affect the whitebalance control, and thus, against the intention of the operator, thedata cannot tell that the object is situated outdoors. To avoid thisproblem, it is required to correctly select data including informationof an object which the operator really intends to pick up. This can beachieved by storing data obtained at a proper time in the middle of arecording operation.

The data recorded in the above-described manner is used as follows. Asdescribed earlier, the conventional video cameras have a problem withthe white balance control in a recording operation just after the powersupply is turned on. If it is known whether an object to be picked up issituated outdoors or situated indoors while being illuminated with anartificial light source, then it is possible to achieve a greatimprovement in the accuracy of the white balance control in an imagepickup operation just after the power supply is turned on. If the lastimage pickup operation was performed outdoors, then there is a highprobability that the next image pickup operation will also be performedoutdoors. Thus, in this invention, if the last image pickup operationwas performed outdoors and if it is turned out that an object has a highilluminance in the next image pickup operation when the power supply isturned on, then it is decided that the image pickup operation is nowbeing performed outdoors.

FIG. 7 is a flow chart illustrating the operation of the discriminationpart 12 relating to the white balance control performed just after thepower supply is turned on. In step 61, the power supply is turned on,and then, immediately after that, the discrimination part 12 determinesthe type of a light source on the basis of the data relating to the lastimage pickup operation stored in the data storing part 42 of thecorrection signal computing part 10. More specifically, in step 62, itis determined whether the last image pickup operation was performedoutdoors. If no, in step 63, it is determined whether the image pickupoperation was performed indoors.

In the above judgement whether the image pickup operation was performedoutdoors, if the data shows that the object picked up in the last timehad an illuminance and a color temperature greater than an outdoorlevel, then it is concluded that the image pickup operation wasperformed outdoors. In this case, it is further determined in step 64whether the current illuminance is greater than the outdoor level. Ifyes, then an outdoor process is performed in step 65. On the other hand,in the above judgement of whether the image pickup operation wasperformed outdoors, if the data shows that the object picked up lasttime had an illuminance and a color temperature lower than the outdoorlevel, then it is determined whether the image pickup operation wasperformed indoors. In this determination, if the data shows that theobject picked up last time had an illuminance and a color temperaturelower than the indoor level, then it is concluded that the image pickupoperation was performed indoors. In this case, it is further determinedin step 67 whether the current illuminance is lower than the outdoorlevel. If yes, then an indoor process is performed in step 68. If theconclusion is “no” in both outdoor and indoor judgements, then theprocess is performed in step 66 such that the image pickup operation iscorrectly performed for either case.

In the outdoor process in step 65, the white balance is adjusted byemploying the white extracting range shown in FIG. 3. Similarly, in theindoor process in step 68, the white balance is adjusted by employingthe white extracting range shown in FIG. 2. In the case where it isimpossible to identify whether the image pickup operation is performedoutdoors or indoors, the white balance is adjusted by employing thewhite extracting range shown in FIG. 4. After the white balanceadjustment, an ordinal operation is performed in step 69.

In the example described above, the light sources are classified intothree types: outdoor illumination, indoor illumination, and unidentifiedillumination. However, the light sources may also be classified into twotypes or four or more types.

FIG. 8 is a block diagram illustrating another embodiment of theinvention in which similar elements to those in FIG. 1 are denoted bythe same reference numerals as those in FIG. 1. In FIG. 8, referencenumeral 43 denotes a viewfinder for observing an object, and referencenumeral 44 is a detector for detecting whether an operator is observingthe viewfinder 43.

In this embodiment, the detector 44 serves as the object detection meansin the previous embodiment. The detector 44 makes the data storing part42 active only when the operator is observing the viewfinder 43, bydetecting, for example, the contact of a part of the human body with afringe of the viewfinder, the reflection of light from the eye, or thelike.

The viewfinder 43 displays a video image being picked up on its screenby electrical or optical means. If the detector 44 detects that anoperator is observing the viewfinder 43, the detector 44 transmits asignal to the data storing part 42. When the operator is observing theviewfinder 43, it can be considered that an object that the operatordesires to pick up is being picked up. Therefore, in this case,information about a state of illumination onto the object is highlyreliable. Thus, the information about a state of illumination is storedin the data storing part 42 when the operator starts or stops observingthe viewfinder 43 or otherwise at a proper time when the operator isobserving the viewfinder 43.

In this way, the information about a state of illumination onto theobject is stored, and this information is used when the power supply isturned on next time so that the white balance is correctly adjustedautomatically for arbitrary object. As can be seen from the abovedescription, the present embodiment of the invention thus providesimprovements in accuracy and reliability in the white balanceadjustment.

What is claimed is:
 1. An image pick-up apparatus comprising: a memorywhich stores information about a brightness of an object picked-up by animage pickup device; and a control device arranged to make size of awhite extracting range, from which an image signal output from saidimage pickup device is extracted as a white image signal, larger in casea current brightness and said brightness about which the information isstored in the memory are not similar, than in case those are similar,and control white balance based on the extracted image, signal.
 2. Anapparatus according to claim 1, wherein said control device controls awhite balance condition by narrowing the white extracting range whensaid control device decides that brightness of the object after turningon a power supply is similar to the previous brightness of the object ascompared to when the brightness are different.
 3. An apparatus accordingto claim 1, further comprising a display device arranged to display animage picked up by said image pickup device, wherein said memory storesthe information of the brightness of the object picked up by said imagepickup device at the time when the image is displayed by said imagepickup device.
 4. An apparatus according to claim 1, wherein saidbrightness is stored in said memory before turning off a power supply.5. An image signal processing method comprising: determining thesimilarity between a current brightness and a brightness about whichinformation is stored in a memory; making a size of a white extractingrange, from which an image signal output from an image pickup device isextracted as a white image signal, larger in case said currentbrightness and said brightness are not similar, than in case those aresimilar; and controlling white balance based on the extracted imagesignal.
 6. A method according to claim 5, wherein a white balancecondition is controlled by narrowing the white extracting range when itis decided that the brightness of the object after turning on a powersupply is similar to the previous brightness of the object as comparedto when the brightness are different.
 7. A method according to claim 5,further comprising displaying a picked up image, wherein the informationof the brightness of the object is stored at the time when the image isdisplayed in said image picking up step.
 8. A method according to claim5, wherein said brightness is stored in said memory before turning off apower supply.